On the LASSO and Its Dual
نویسندگان
چکیده
Proposed by Tibshirani (1996), the LASSO (least absolute shrinkage and selection operator) estimates a vector of regression coefficients by minimising the residual sum of squares subject to a constraint on the l-norm of coefficient vector. The LASSO estimator typically has one or more zero elements and thus shares characteristics of both shrinkage estimation and variable selection. In this paper we treat the LASSO as a convex programming problem and derive its dual. Consideration of the primal and dual problems together leads to important new insights into the characteristics of the LASSO estimator and to an improved method for estimating its covariance matrix. Using these results we also develop an efficient algorithm for computing LASSO estimates which is usable even in cases where the number of regressors exceeds the number of observations.
منابع مشابه
Lasso Screening Rules via Dual Polytope Projection
Lasso is a widely used regression technique to find sparse representations. When the dimension of the feature space and the number of samples are extremely large, solving the Lasso problem remains challenging. To improve the efficiency of solving large-scale Lasso problems, El Ghaoui and his colleagues have proposed the SAFE rules which are able to quickly identify the inactive predictors, i.e....
متن کاملDifferenced-Based Double Shrinking in Partial Linear Models
Partial linear model is very flexible when the relation between the covariates and responses, either parametric and nonparametric. However, estimation of the regression coefficients is challenging since one must also estimate the nonparametric component simultaneously. As a remedy, the differencing approach, to eliminate the nonparametric component and estimate the regression coefficients, can ...
متن کاملc-LASSO and its dual for sparse signal estimation from array data
We treat the estimation of a sparse set of sources emitting plane waves observed by a sensor array as a complex-valued LASSO (c–LASSO) problem where the usual l1-norm constraint is replaced by the l1-norm of a matrix D times the solution vector. When the sparsity order is given, algorithmically selecting a suitable value for the c–LASSO regularization parameter remains a challenging task. The c...
متن کاملA New Combined Approach for Inference in High-Dimensional Regression Models with Correlated Variables
We consider the problem of model selection and estimation in sparse high dimensional linear regression models with strongly correlated variables. First, we study the theoretical properties of the dual Lasso solution, and we show that joint consideration of the Lasso primal and its dual solutions are useful for selecting correlated active variables. Second, we argue that correlation among active...
متن کاملThe Double-Barrelled LASSO
We present a new method which solves a double-barelled LASSO in a convex least squares approach. In the presented method we focus on the scenario where one is interested in (or limited to) a primal (feature) representation for the first view while having a dual (kernel) representation for the second view. DB-LASSO minimises the number of features used in both the primal and dual projections whi...
متن کامل